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ABSTRACT 


In  this  note  we  discuss  Newton's  method  in  a  setting 
somewhat  more  restrictive  than  customary.  In  this  setting,  however, 
we  claim  to  have  proved  superlinear  convergence  of  the  Newton  process 
without  assuming  twice  differentiability  or  Lipschitz  continuity  of 
the  first  derivative  of  the  operator.  A  further  feature  is  that 
the  iteration  to  be  discussed  is  not  initially  but  is  eventually  the 
Newton  process.  With  this  feature  global  rather  than  local  con¬ 


vergence  is  achieved. 


The  literature  on  Newton's  method  after  the  fundamental 


paper  of  Kantorovich  [1]  is  large.  Pertinent  to  this  note  is 
the  work  of  Schrdder  [2],  in  which  the  first  derivative  is 
assumed  to  be  Lipschitz  continuous  and  quadratic  convergence 
ensues.  Another  related  paper  showing  the  effectiveness  of 
Newton's  method  with  once  differentiability  may  be  found  in 
the  literature,  but  this  claim  was  later  retracted  by  the  author. 
In  Schrdder  [2],  a  long  list  of  references  to  Newton's  method  may 
be  found. 


Let  Q  be  a  differentiable  non-linear  operator  with  domain 
and  range  in  a  Hilbert  space  H.  Given  xq  arbitrarily  in  H, 
set  f (x)  =  /][Q(xq  +  t(x  -  Xq),  x  -  x0]dt,  and  S  -  {x  e  H:  f (x)  <  0}. 
Let  f'(x,h)  denote  the  F-dif f erential  (F  *  Frdchet)  of  f.  Take 
in  H  so  that  [Vf(x),h]  ■  f'(x,h)  for  all  h  c  H.  Let 
Q'(x,h)  denote  the  F-diff erential  of  Q.  Let  Q'(x,h)  *  J(x)h. 

Thus  for  each  x,  J(x)  is  a  linear  operator  on  H. 

Lemma. [ 31  Assume  J(")  is  continuous  on  S  and  J(x)  is 
self-adjoint  for  all  x  c  S.  Then  Q(x)  *  Vf(x). 

Proof .  We  shall  prove  that  for  x  t  S  and  h  c  H, 
f'(x,h)  =  (Q(x),h).  By  calculation  f'(x,h)  = 

•  Z1 {  [Q' (x0  +  t(x  -  x0),  th),  X  —  Xq ]  +  l Q ( X 0 )  +  t (x  -  x0),  h]}dt, 
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and  [Q(x),h]  -  /J  Q(x0  +  t(x  -  x0),th]dt  - 
■  /J  UQ'(x0  +  t(x  -  x0),  x  -  xq)  ,  th]  +  [  Q  ( x  o  +  t(x  -  x0),h]}dt  - 
f'(x,h).  Q.E.D. 


For  the  next  theorem  we  set  A(x,p)  *  f(x)  -  f(x  -  PJ-1(x)Q(x)) 
and  g(x,p)  »  A(x,p) /p [Q(x) J"1 (x)Q(x) ] .  We  shall  abbreviate 
Q(xR),  J(xR),  etc.,  by  Qk  and  JR,  respectively. 


Theorem.  Assume  J(*) 

assume  J(x)  self-adjoint  and 

all  h  c  H  and  some  y  >  0. 

p.  is  chosen  so  that  for  6  < 
k 

Pk  ■  1  ,  if  possible.  Then: 


continuous  on  S.  For  each  x  e  S 
y|  | h |  |2  <_  [h, J(x)h]  <_  A|  | h |  |2  for 

Set  xk+l  '  xk  '  pkJklQk’  Here 
h,  o  <  6  <  g(xk,pk)  1  1  -  6,  with 


(a)  For  arbitrary  x°  the  sequence  {x^}  converges  to  the 
unique  root  of  Q. 

(b)  There  exists  a  number  N  such  that  if  k  >  N  then 


(c)  The  convergence  of  <xk)  is  superlinear — that  is  to 
say  it  is  faster  than  any  geometric  progression. 


proof .  We  show  first  that  if  Q(xk)  ?  0  and  xk  e  S 
then  pk  can  be  chosen  as  claimed.  By  Taylor's  theorem 
A(x,p)  *  f(x)  -  f (x  -  pJ_1 (x))Q(x))  «  p [Q(x) ,J_1 (x)Q(x) ]  - 

-  p2[J_1  (x)Q(x)  ,JU(p>)  J"1  (x)Q(x)  ]/2  where  S(p)  lies  "between" 
x  and  x  -  pJ-1(x)Q(x).  Hence  g(x,p)  = 

1  -  p [ J“ 1 (x)Q(x)J(C(p)J_1(x)Q(x) ]/2[Q(x)J_1 (x)Q(x)].  It  follows 
that  g ( x , • )  is  continuous  at  each  p  for  which  £(p)  c  S.  In 
particular  g(x,0)  *  J .  If  S  were  bounded,  then  we  observe 
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that  there  would  exist  a  least  positive  number,  say  p,  for  which 
g(x,p)  *  0,  with  p  <  p  implying  f,(p)  t  S.  Hence  if  S  is 
bounded,  there  exists  p  >  0  such  that  A  g(x,p)  1  -  A. 

Suppose  then  S  were  unbounded.  If  so,  there  exists  a  sequence 
{x^}  c  S  such  that  {  |  |x^|  |  }  t  o",  while  ffx^)  <  0  for  all  k. 

By  Taylor ' s  theorem  if  u  c  S,  f  (x^)  —  f  (u)  +  I  I  xk  ~  u  |  | ( ( |  | x^  “  u||u/2) 
-  |  |  V f  ( u )  |  |  ]  showing  that  ffx^)  —  0  for  ^ar8e  k.  Thus  the  numbers 
p^  are  well  defined. 

Observe  next  that  A(x,o)  [Q(x)J-1  (x)Q(x)  ]  [1  -  p  A 2  /  p 2  ] 

for  all  p  such  that  the  line  segment  [x,  x  -  pj"1 (x)Q(x) ] 

belongs  to  S.  To  prove  this  we  apply  the  inequalities 

p|  |h|  |2  1  [h ,  J  (x)  h  ]  A  |  j  h  |  |  2  and  j|h||2/X  £  [h,J_1(x)h]  _<  ||h||2/p 

to  the  above  Taylor  expansion  of  A(x,p),  It  follows  that 

g(x^,p^)  _>  [1  -  pRA2  /  2p2  ]  and  therefore  _>.  26y2/A2.  Thus 

A(xR,pk)  1  2|  |  Q^l  |  26y2/A3.  Suppose  now  that  (  |  |  QjJ  |  0.  Then 

a  subsequence  of  {A(x^,p^)}  I  -°°,  and  thus  {f(x^)}  I  -®. 

Observe  that  f  is  continuous  on  S  and  that  since  f  is  convex, 

the  set  S  *  {x  c  5:  f(x)  <  m}  is  closed  and  convex  for  each  m. 
m  — 

Thus  the  set  is  weakly  closed  and  the  function  f  is  weakly 

lower  semicontinuous .  Since  S  is  weakly  compact,  f  achieves  a 
minimum  on  S,  which  contradicts  that  (f(x^)}  I  -°°.  It  follows 
therefore  that  MQ^II  °>  and  a  limit  L.  Next  we 

prove  that  {x^}  is  Cauchy.  By  Taylor's  theorem,  if  s  >  k, 
f(xS)  -  f(xk)  _>  [Qk>xS  “  xk]  +  u  |  j  xS  -  xkj):/2.  Since  S  is 
I  I xS  ~  xk|  |  B  where  B  is  the  diameter  of  S. 


bounded 


Thus 
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|  |xS  -  x^||2  <_  (f(xS)  -  f(x^)  +  |  |QjJ  | B) 2/ jj .  Since  {f(x^)} 

s  k 

is  Cauchy  and  |  |Q  |  |  -♦  0,  { x  }  is  Cauchy.  Thus  {x  }  -*■  z 

and  Q(z)  *  0.  Suppose  now  that,  z  is  not  unique,  so  that 
Q(z  j )  -  Q(z2)  *  0.  Thus  f  (z  j )  -  f(z2)  >,  u|  |zj  -  z  2  |  |  2  , 
f(z2)  -  f  (z ! )  >_  u|  |z2  -  z  i  I  1 2 ,  and  u||z2  -z1j|2_<f(z2)  -  f (zt) 

£  -u  i  |z  2  -  z ] |  j  2 ,  which  is  a  contradiction  unless  z i  *  z  2 • 

To  prove  (b),  we  observe  that  by  setting  J(£(p))  ■ 

*  J ( x)  +  J(£(p))  -  J(x)  that  g(x,p)  -  1  -  p/2  - 

-  p[J_1(x)Q(x) ,(J(£(p))  -  J(x))J— 1 (x)Q(x) ]/2[Q(x) ,J—1 (x)Q(x) ] . 

Thus  j  g  (x ,  p )  -  1  +  p/2|  <_  Xp  |  |  J(C(p) )  -  J(x)||/2u2.  Since 

{ xR }  is  Cauchy,  {||xk+1  -  xk |  j  }  and  {  |  U(Pk)  -  xR|  |  }  converge 

to  0.  Indeed  if  £,(p  )  =  £  then  { xq , £q »xi » £ l » *  *  *  * )  is  also 

k  k 

a  Cauchy  sequence;  and  it,  together  with  (z)  is  a  compactum. 
Consequently  on  this  compactum  J  is  uniformly  continuous. 
Therefore  | |j(£k>  -  J(xk>  |  |  -*>  0.  Let  -  A |  | -  J(xk>||/2y2 
and  assume  that  k  is  taken  sufficiently  large  so  that  £k  < 

Then  |g(xk,pk>  -  1  +  ok/2|  <_  ekPk  implies 


1 '  8(Vpk} 

h  + 


-pk- 


-  «(xk-°k) 
h  -  e. 


Thus  eventually  ok  cai.  be  any  point  which  satisfies 

26  <  pk  <_  2(1  -  6).  Therefore  since  26  <  1,  the  choice  ck  =  1 

will  be  eventually  feasible. 


To  nrove  (c)  we  write  x,  , ,  -  a1 

k+1 


*k ' 2  ‘  V1 


xk  -  i'.  -  PkJ_1  (xk)J(xk)(xk  -  5$  +  PkJ-1  (xk)  [J(xk)  (x 


By  hypothesis  Q  is  Frdchet  differentiable  at  x  , 

K 


(xk)Q(xk)  * 
k  -  z )  -  Q(xk)]. 

consequently 
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given  c  >  0  there  exists  6  >  0  such  that  | ) Q ( z)  -  Q(x^) 

-  J(xk)(z  -  xk)  I  I  <  t  | lz  -  xk| I,  whenever  |  |*  -  xR| |  <  6. 
Thus  we  choose  k  so  large  that  this  latter  inequality  la 
fulfilled  we  get  j  |xk+1  -  z| |  I |xfc  -  z| | (1  -  e/u) . 


If  furthermore  k  _>  N, 
the  superlinear  convergence. 


Xk+1  '  Z 


1  1 1  I xk  -  z  |  |  /u,  showing 
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